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Abstract 

Irregular pyramids are made of a stack of successively reduced graphs embedded 
in the plane. Such pyramids are used within the segmentation framework to encode 
a hierarchy of partitions. The different graph models used within the irregular pyra- 
mid framework encode different types of relationships between regions. This paper 
compares different graph models used within the irregular pyramid framework ac- 
cording to a set of relationships between regions. We also define a new algorithm 
based on a pyramid of combinatorial maps which allows to determine if one region 
contains the other using only local calculus. 
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1 Introduction 



Graphs play an important role in computer vision and pattern recognition 
since the birth of these fields. Graphs are used along the overall process from 
the stimuli to the interpretation task: hierarchical and non-hierarchical data 
structures for image segmentation, graph matching for pattern recognition, 
graph clustering for structural classification, and computation of a median 
graph [1] for learning the structural properties of models. 

Graphs are thus used both for low level image processing and high level tasks. 
Different type of graphs being used for different types of applications. How- 
ever, in many computer vision tasks, the low image segmentation stage cannot 
be readily separated from higher level processing. On the contrary, the seg- 
mentation algorithms should often extract fine information about the partition 
in order to guide the segmentation process according to the high level goal. 
This information may be used to compare isolated regions or some local con- 
figuration of regions to a model. There is thus a need to design graph models 
for image segmentation which can be both efficiently updated and allow to 
extract fine information about the partition. 

1.1 Relating Regions 

Different formalisms such as the RCC-8 defined by Randel [2] or the relation- 
ships defined by K. Shearer et al. [3] in the context of graph matching may 
be used to relate the regions of a partition. Within the particular context of 
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image segmentation, the following relationships may be defined from these two 
models: meets, contains, inside: 

(1) The meets relationship means that two regions share at least one com- 
mon boundary. The different models used to encode partitions either 
encode the existence of this common boundary or create one relationship 
for each boundary between two regions. We denote these two types of 
encodings meets_exists and meets-each. The ability of the models to re- 
trieve efficiently a given common boundary between two regions is also 
an important feature of these models. 

(2) The relationship A contains B expresses the fact that region B is inside 
region A. For example, the background of the road sign in Fig. 1(a) 
contains the upside arrow. 

(3) The inside relationship is the inverse of the contains relation: A region 
B inside A is contained in A. 

One additional relationship not directly handled by the models of Shearer and 
Randel may be defined within the hierarchical segmentation scheme. Indeed 
within such a framework a region defined at a given level of a hierarchy is 
composed of regions defined at levels below. 

The following relationships may thus be deduced from the relationships defined 
by Shearer and Randel: The meets-exists, meets-each, contains, inside and 
composed of. Note that unlike meets relationships, the contains and inside 
relations are asymmetric. A contains or inside relation between two regions 
allows thus to characterize each of the regions sharing this relation. 
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1.2 Region Adjacency Graph 



One of the most common graph data structure, within the segmentation frame- 
work is the Region Adjacency Graph (RAG). A RAG is defined from a parti- 
tion by associating one vertex to each region and by creating an edge between 
two vertices if the associated regions share a common boundary. A RAG cor- 
responds thus to a simple graph without any double edge between vertices 
nor self-loop. Within a non-hierarchical segmentation scheme the RAG model 
is usually applied as a merging step to overcome the over-segmentation pro- 
duced by the previous splitting algorithm [4]. Indeed, the existence of an edge 
between two vertices denotes the existence of at least one common boundary 
segment between the two associated regions which may thus be merged by re- 
moving this segment. Within this framework, the edge information may thus 
be interpreted as a possibility to merge the two regions identified by the ver- 
tices incident to the edge. Such a merge operation implies to collapse the two 
vertices incident to the edge into one vertex and to remove this edge together 
with any double edge between the newly created vertex and the remaining 
vertices. 

The RAG model encodes thus only the existence of a common edge between 
two regions (the meets_exists relationship). Moreover, the existence of a com- 
mon edge between two vertices does not provide enough information to dif- 
ferentiate a meets relationship from a contains or inside one. This drawback 
is illustrated on an ideal segmentation of two roadsigns (Fig. 1) which are 
encoded by a same RAG. The road sign (a) defines two nested contains rela- 
tionships. Indeed, the white border contains the background which contains 
itself the symbol. On the other hand the road sign (b) corresponds to two 
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meets relationships between the central region and its two white neighbors. 

1.3 Combinatorial Maps 

A 2D combinatorial map may be understood as a planar graph encoding ex- 
plicitly the orientation in the plane. Each connected component of a partition 
(a connected set of regions) may be encoded by a 2D combinatorial map up 
to an homeomorphism [5]. One of the main insight of such models compared 
to a RAG data structure is their ability to be efficiently updated after both 
split and merge operations. 

The combinatorial map formalism allows to encode each connected boundary 
between two regions by one edge. The models based on combinatorial maps 
encode thus the meets-each relationship. However, within the combinatorial 
map framework two connected components of a partition will be encoded by 
two combinatorial maps without any information about the respective po- 
sitioning of the two components. The models based on combinatorial maps 
have thus designed additional data structure like the inclusion tree [6] or the 
Parent-Child relationships [5,7] to encode the contains and inside relation- 
ships. Using these models any modification of the partition implies to update 
both the combinatorial maps and the additional data structures. 

1.4 Segmentation Hierarchies 

Data structures used within the hierarchical segmentation framework encode 
a stack of partitions successively simplified by region merging. Irregular pyra- 
mids models introduced by Meer and Montanvert [8] encode each partition 
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by a graph where each vertex is associated to one region. At each level of the 
pyramid a region is obtained by the merge of a connected set of regions at the 
level below. The resulting region is called the parent of the merged regions. 
These last regions correspond to the child of the region at the level above. The 
models based on the irregular pyramid framework encode thus naturally the 
composed of relationship. In order to preserve the efficiency of a hierarchical 
data structure, the size of models encoding each partition of the hierarchy 
must be strictly decreasing according to the level. This last constraint forbids 
the use of an additional data structure similar to the structures used for com- 
binatorial maps models in order to store contains and inside relationships. 
Indeed, contains and inside relationships between regions may both be re- 
moved and created along the different levels of the pyramid. The use of such 
an additional data structure may thus violate the strictly decreasing size of 
the models according to the levels. 

1.5 Overview 

The aim of this paper is twofold: We firstly provide an introduction to the 
main data structures used within the hierarchical segmentation framework 
according to the set of relationships previously defined (Section 1.1). Secondly, 
we present an efficient computation of the contains and inside relationships 
within the irregular pyramid framework. The remaining of this paper is thus 
organized as follows: Section 2 presents two models belonging to the irregular 
pyramid framework together with their properties relative to the relationships 
previously defined. Section 3 describes the combinatorial map model and its 
main properties. Section 4 describes the construction scheme and the main 
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properties of a pyramid of combinatorial maps : A combinatorial Pyramid. 
Finally, Section 5 presents one algorithm computing the contains and inside 
information using only local calculus. 

2 Simple and Dual Graph Pyramids 

The irregular pyramids are defined as a stack of successively reduced graphs, 
each graph being built from the graph below by selecting a set of vertices 
named surviving vertices and mapping each non-surviving vertex to a surviv- 
ing vertex [9,8]. Using such a framework, the graph Gi + i = (VJ+i, -Ei+i) defined 
at level 1 + 1 is deduced from the graph defined at level / by the following steps: 

(1) Select the vertices of among VJ. These vertices are the surviving 
vertices of the decimation process, Vi+i C V\. 

(2) Each non-surviving vertex connects to a surviving vertex by one edge 
of G\. The set of vertices attached to each surviving vertex defines a 
partition of V\. 

(3) Define the adjacency relationships between the vertices of in order 
to define Ei + \. 

2.1 Simple graph Pyramids 

In order to obtain a decimation ratio greater than 1 between two successive 
levels, Meer [9] imposes the following constraints on the set of surviving ver- 
tices: 
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V(v,v')eV? +1 : (v,v')^E t 



(1) 

(2) 



Constraint (1) insures that each non- surviving vertex is adjacent to at least 
a surviving vertex. Constraint (2) insures that two adjacent vertices cannot 
both survive. These constraints define a maximal independent set (MIS) [9,8]. 

Given the set of surviving vertices, different methods [8,10] may be used to link 
each non-surviving vertex to one of its surviving neighbor. For example, Mon- 
tanvert [8] attaches each non-surviving vertex to its closest surviving neighbor 
according to a difference between the outcome of a random variable attached 
to each vertex. The set of non-surviving vertices connected to a surviving 
vertex defines its reduction window and thus the parent child relationship 
between two consecutive levels. 

The final set of surviving vertices defined on V\ corresponds to the set of 
vertices V/ + i of the reduced graph G7+1 = (VJ+i, £7+1). The set of edges Ei+i 
of Gi+i is defined by connecting by an edge in any couple of surviving 
vertices having adjacent children. 

Two surviving vertices are thus connected in G7+1 if they are connected in Gi 
by a path of length lower or equal than 3. Two reduction windows adjacent 
by more than one path of length lower or equal than 3 will thus be connected 
by a single edge in the reduced graph. The stack of graphs produced by the 
above decimation process is thus a stack of simple graphs each simple graph 
encoding only the existence of one common boundary between two regions 
(the meeets-exists relationship). Moreover, as mentioned in Section 1.2 the 
RAG model which corresponds to a simple graph does not allow to encode 
contains and inside relationships. 
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2.2 Construction of Dual Graph Pyramids 

The dual graph pyramids introduced by Willersinn and Kropatsch [11] use 
an alternative construction scheme. Within the dual graph pyramid frame- 
work the reduction process is performed by a set of edge contractions. The 
edge contraction collapses two adjacent vertices into one vertex and removes 
the edge. Many edges except self-loops can be contracted independently of 
each other and also in parallel. In order to avoid contracting a self-loop these 
edges should not form cycles, e.g. form a forest. This set of edges is called a 
contraction kernel. 

The contraction of a graph reduces the number of vertices while maintaining 
the connections to other vertices. As a consequence some redundant edges may 
occur. These edges belong to one of the following categories: 

• Redundant double edge: These edges encode multiple adjacency relation- 
ships between two vertices and define degree two faces. They can thus be 
characterized in the dual graph as degree two dual vertices. In terms of parti- 
tion's encoding these edges correspond to an artificial split of one boundary 
between two regions. 

• Empty self-loop: These edges correspond to a self-loop with an empty inside. 
These edges define thus degree one faces and are characterized in the dual 
graph as degree one vertices. Such edges encode artificial inner boundaries 
of regions. 

Both double edges and empty self-loops do not encode relevant topological 
relations and can be removed without any harm to the involved topology [11]. 
The removal of such edges is called a dual decimation step and the set of 
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removed edges is called a removal kernel. Such a kernel defines a forest of the 
dual graph. 

2.3 Dual Graph Pyramids and multiple boundaries 

Given one tree of a contraction kernel, the contraction of its edges collapses 
all the vertices of the tree into a single vertex and keeps all the connections 
between the vertices of the tree and the remaining vertices of the graph. The 
multiple boundaries between the newly created vertex and the remaining ver- 
tices of the graph are thus preserved. Each graph of a dual graph pyramid 
encodes thus the meets_each relationships. This property is not modified by 
the application of a removal kernel which only removes redundant edges. 

2.4 Dual Graph Pyramids and the inside relationship 

Due to the forest requirement, the encoding of the adjacency between two re- 
gions one inside the other will be encoded by two edges (Fig. 2): One edge en- 
coding the common border between the two regions and one self-loop incident 
to the vertex encoding the surrounding region. One may think to characterize 
the inside relationship by the fact that the vertex associated to the inside 
region should be surrounded by the self-loop. However, as shown by Fig. 2(c) 
one may exchange the surrounded vertex without modifying the incidence re- 
lationships between both vertices and faces. Two dual graphs being defined 
by these incidence relationships one can exchange the surrounded vertex with- 
out modifying the encoding of the graphs. This last remark shows that the 
inside/contains relationships cannot be characterized locally within the dual 
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graph framework. 

3 Combinatorial maps 

Combinatorial maps and generalized combinatorial maps define a general 
framework which allows to encode any subdivision of nD topological spaces 
orientable or non-orientable with or without boundaries. Recent trends in com- 
binatorial maps apply this framework to the segmentation of 3D images [12,13] 
and the encoding of 2D [14,15] and nD [16] hierarchies. 

The remaining of this paper will be based on 2D combinatorial maps which will 
be just called combinatorial maps. A combinatorial map may be deduced from 
a planar graph by splitting each edge into two half edges called darts. An edge 
connecting two vertices is thus composed of two darts, each dart belonging 
to only one vertex. The relation between two darts d\ and c?2 associated to 
the same edge is encoded by the permutation a which maps d\ to a?2 and di 
to d\. The permutation a is thus an involution and its cycles 2 are denoted 
by a*(d) for a given dart d. These cycles encode the edges of the graph. The 
sequence of darts encountered when turning around a vertex is encoded by the 
permutation a. Using a counter-clockwise orientation, the cycle cr*(d) encodes 
the set of darts encountered when turning counter-clockwise around the vertex 
encoded by the dart d. A combinatorial map can thus be formally defined by 
G = (T>,a,a), where T> is the set of darts and a, a are two permutations 

2 the cycle of a dart d associated to a permutation it on the set of darts is the 
sequence (d, Tr(d), vr 2 (d), . . . , ir n (d)) with ir n (d) = d. Since the set of darts is finite n 
is defined for any dart and any permutation ir. The tt orbit of a dart d correponds 
to the same set of darts as its cycle but without any ordering between darts. 
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defined on T> such that a is an involution. 3 

Given a combinatorial map G = (T>, a, a), its dual is defined by G = (T>, </?, a) 
with <p = a o a. The cycles of the permutation ip encode the set of darts 
encountered when turning around a face of G. 

We can state one of the major difference between a combinatorial map and 
an usual graph encoding of a partition. Indeed, a combinatorial map may be 
seen as a planar graph with a set of vertices (the cycles of a) connected by 
edges (the cycles of a). However, compared to an usual graph encoding a 
combinatorial map encodes additionally the local orientation of edges around 
each vertex thanks to the order defined within each cycle of a. 

Fig. 3 illustrates the encoding of a 3 x 3 4-connected discrete grid by a combi- 
natorial map G. The involution a is implicitly encoded by the sign in Fig. 3(a) 
and (b). We have thus a(d) = —d for all darts on these figures. 

Since G encodes a planar sampling grid, each vertex of G (Fig. 3(b)) is as- 
sociated to a corner of a pixel. For example, the top left pixel of the 3x3 
grid is encoded by the a cycle c*(l) = (1, 13, 24, 7) (top left vertex and square 
in Fig. 3(a) and (b)). The top-left corner of this pixel is encoded by the tp 
cycle: Lp*(2A) = (24,-13) (top left dual vertex of Fig. 3(b)). Moreover, each 
dart may be understood in this combinatorial map as an oriented crack, i.e. 
as a side of a pixel with an orientation. For example, the dart 1 in Fig. 3(b) 
encodes the right side of the upper-left pixel oriented from bottom to top. The 
f, a and a cycles of a dart may thus be respectively understood as elements 
of dimensions 0, 1 and 2. 

3 7r is an involution on T> if tt o ir(d) = d for any dart d in T> 
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Each dart of a combinatorial map G, encoding a planar sampling grid may 
thus be interpreted as an oriented crack and associated to a point encoding 
the coordinates of a pixel's corner and one move encoding the orientation on 
the crack associated to the dart. 

4 Embedding and Orientation 

As in the dual graph pyramid scheme [17] (Section 2) a combinatorial pyramid 
is defined by an initial combinatorial map successively reduced by a sequence 
of contraction or removal operations. The initial combinatorial map encodes 
a planar sampling grid (Section 3) or a first segmentation and the remain- 
ing combinatorial maps of a combinatorial pyramid encode a stack of image 
partitions successively reduced. Such combinatorial maps are thus embedded 
(Section 4.3). As mentioned in Section 3 a combinatorial map may be under- 
stood as a dual graph with an explicit encoding of the orientation of the edges 
incident to each vertex. This explicit encoding of the orientation is preserved 
within the combinatorial pyramid using contraction and removal operations 
equivalent to the operations used for dual graphs but which preserve the orien- 
tations of edges around the vertices of the reduced combinatorial maps [14,15]. 

Contraction operations are controlled by contraction kernels (CK). The re- 
moval of redundant edges is performed as in the dual graph reduction scheme 
by a removal kernel. This kernel is however decomposed in two sub-kernels : 
A removal kernel of empty self-loops (RKESL) which contains all darts inci- 
dent to a degree 1 dual vertex and a removal kernel of empty double edges 
(RKEDE) which contains all darts incident to a degree 2 dual vertex. These 
two removal kernels are defined as follows : The removal kernel of empty self- 
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loops RKESL is initialized by all self-loops surrounding a dual vertex of degree 
1. RKESL is further expanded by all self- loops that contain only other self- 
loops already in RKESL until no further expansion is possible. For the removal 
of empty double edges RKEDE we ignore all empty self-loops in RKESL in 
computing the degree of the dual vertex. Note that the successive application 
of a RKESL and a RKEDE is equivalent to the application of a removal kernel 
defined within the dual graph framework. Both contraction and removal oper- 
ations defined within the combinatorial pyramid framework are thus defined 
as is the dual graph framework but additionally preserve the orientation of 
edges around each vertex. Further details about the construction scheme of a 
Combinatorial Pyramid may be found in [14]. 

4-1 What is inside ? 

Combinatorial pyramids are thus built using the same framework as dual 
graphs pyramids. The use of a contraction kernel within the construction 
scheme of a combinatorial pyramid allows to encode multiple adjacency be- 
tween regions thanks to multiple edges between their associated vertices. 
Therefore, as in the dual graph framework, combinatorial pyramids preserve 
the meets_each relationship (Section 2.3). Note that the explicit encoding of 
the orientation within the combinatorial pyramid framework does not interfere 
with this last property. 

Moreover, as in the dual graph framework (Section 2.4), an inside relationship 
between two regions is encoded by two edges: one edge encodes the common 
border between the two regions while the other encodes a self-loop incident to 
the vertex associated to the surrounding region. Let us consider the example 
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already used for dual graph pyramids (Section 2.4, Fig. 2). Fig. 4 shows the 
encoding of the ideal segmentation of the road sign using combinatorial maps. 
As shown by Fig. 4 (b) and (c), one can exchange the surrounded vertex with- 
out changing the order of the darts around the vertex cr*(l). Therefore, the 
two drawings shown in Fig. 4(b) and (c) are encoded by the same combina- 
torial map. One cannot determine from the formally specified combinatorial 
map which part is inside and which is contained. This ambiguity may also 
be characterized using the cycle cr*(l) of the vertex incident to the self-loop. 
Indeed, this cycle is equal to cr*(l) = (1,2, —1,3). Since (1, —1) is a self-loop 
the neighbors of <x*(l) have in their a cycles the a successors of the two darts 
2 and 3. The ambiguity about the drawing of the self- loop is characterized 
on the cycle er*(l) by the fact that we can not deduce from this cycle if the 
dart 2 is between 1 and —1 or if on the contrary 3 is between —1 and 1. This 
ambiguity arises thus because the two darts 1 and —1 play a symmetric role in 
<r*(l). We can thus state the two following points from the above discussion: 

(1) Combinatorial pyramids preserve the meets_each relationship. 

(2) An inside relationship 'A inside B' is always associated with a self-loop 
incident to B. However, a non-redundant self- loop at B does not always 
identify the inside region. 

4-2 Implicit encoding of a combinatorial pyramid 

Let us consider an initial combinatorial map Go = (£*> o~, a) and a sequence of 
kernels K\, . . . , K n successively applied on Go to build the pyramid. Each com- 
binatorial map Gj = (ST>i,o~i,ati) is defined from Gj_i = (<S£>j_i, (7j_i, aj_i) 
by the application of the kernel Ki on Gj_i and the set of darts ST>i is equal 
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to <SZ>j_i \ Ki. We have thus: 

SV n+1 C SV n C ... SVt C V (3) 

The set of darts of each reduced combinatorial map of a pyramid is thus 
included in the base level combinatorial map. This last property allows us to 
define the two following functions: 

(1) one function state from {1, . . . , n} to the states {CK, RKESL, RKEDE} 
which specifies the type of each kernel. 

(2) One function level defined for all darts in T> such that level(d) is equal 
to the maximal level where d survives: 

\/d £ V leveled) = Max{i £ {1, . . . , n + 1} | d £ SXVi} 

a dart d surviving up to the top level has thus a level equal to n + 1 . Note 
that if d £ Ki, i £ {1, . . . , n} then leveled) = i. 

We have shown [10,14] that the sequence of reduced combinatorial maps 
Go, ... , G n+ i may be encoded without any loss of information using only the 
base level combinatorial map Go and the two functions level and state. Such 
an encoding is called an implicit encoding of the pyramid. 

The receptive field of a dart d £ ST>i corresponds to the set of darts reduced 
to d at level i [14,10]. Using the implicit encoding of a combinatorial pyramid, 

the receptive field RFi{d) of d £ ST>i is defined as a sequence d\ d q of 

darts in V by d\ = d, d<i = <Jo(d) and for each j in {3, . . . , q} : 



tpo(dj-i) if state(level(dj-i)) = CK 

(4) 

o-o(eZj-i) if state(level(dj-i)) £ {RKEDE, RKESL} 



1(3 



The dart d q is defined as the last dart of the sequence which have been con- 
tracted or removed below the level i. Therefore, the successor of d q according 
to equation 4, d q+ i satisfies level(d q+ i) > i. Moreover, we have shown [14,10] 
that d, d q and d q+ \ are additionally connected by the two following relation- 
ships: 

(7i{d) = d q+1 and a^d) = a (d q ) (5) 

Note that these two last relationships allow to retrieve any reduced combina- 
torial map of the pyramid from its base. 

The implicit encoding of combinatorial pyramids is thus based on the fact that 
the set of darts of any reduced combinatorial map is included in the initial 
combinatorial map (equation 3). The two functions state and level which 
are based on this property allow to encode the whole sequence of reduced 
combinatorial map without loss of information [14,10]. 

4-3 Dart's embedding and Segments 

In the RAG a region corresponds to a vertex and two regions are connected 
by an edge if the two regions share a boundary. In the Combinatorial Map, 
vertices and edges correspond to a and a cycles respectively. Therefore, each 
dart d G ST>i encodes a boundary between the regions associated to <J*(d) 
and a*(ai(d)). Moreover, in the lower levels of the pyramid the two vertices 
of an edge may belong to a same region. We call the corresponding bound- 
ary segment an internal boundary in contrast to an external boundary which 
separates two different regions of a RAG. The receptive field of d at level i 
(RFi(d)) contains both darts corresponding to this boundary and additional 
darts corresponding to internal boundaries. The sequence of external bound- 
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ary darts contained in RFi(d) is denoted by dRFi(d) and is called a segment. 
The order on dRFi(d) is deduced from the receptive field RFi(d). Given a dart 
d E ST>i, the sequence dRF^d) = d x , . . . , d q is retrieved by [10]: 



d\ = d and Vj E {1, . . . , q — 1} dj +1 = Lp§ (a (d,)) 



(6) 



The dart d q is the last dart of dRFi (d) which belongs to a double edge kernel. 
This dart is thus characterized using equation 5 by d q = ao{ai{d)). Note that 
each dart of the base level corresponds to an oriented crack (Section 3). A seg- 
ment corresponds thus to a sequence of oriented cracks encoding a connected 
boundary between two regions [10]. 

The value rij is defined for each j E {1, . . . ,q — l}by: 



A segment may thus be interpreted as a maximal sequence, according to equa- 
tion 6, of darts removed as double edges. Such a sequence connects two darts 
(d and ao{d q ) = oti{d)) surviving up to level i. The retrieval of the boundaries 
using equations 6 and 7 is one of the major reason which lead us to distinguish 
empty self-loop removal kernels and double edges. 

Let us additionally note that if Go encodes the 4-connected planar sampling 
grid, each if cycle is composed of at most 4 darts (Fig. 3(b)). Therefore, the 
computation of dj + i from dj (equation 6) requires at most 4 iterations and the 
determination of the whole sequence of cracks composing a boundary between 
two regions is performed in a time proportional to the length of this boundary. 



Min{k E IN* | state(level(ip%(a (dj)))) = RKEDE}. 



(7) 
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4-4 Computing Segment's Orientation 



As mentioned in Section 3, each oriented crack associated to an initial dart 
may be encoded by the position of its starting point and one move. The move 
of an initial dart d is denoted by move(d). If the initial combinatorial map 
G encodes a square grid, the move associated to each dart belongs to the 
set {right, up, left, down}. These initial moves are encoded using Freeman's 
codes: right,up, left and down are numbered from to 3. The angle between 
two moves mi and m 2 denoted by (mi, m 2 )~ is then defined as: (mi— 7712) mod 4 
where mod corresponds to the operator modulo. This angle is thus equal to: 
+1 if the two oriented cracks define a clockwise 90° turns, -1 if the two oriented 
crack define a counter-clockwise 90° turns, if the two oriented crack corre- 
spond to a same move and 2 if the two oriented cracks correspond to opposite 
moves. Note that these angles may be associated to the RULI code (Right turn, 
U turn, Left turn and Identical) defined by Fermiiller and Kropatsch [18]. In- 
deed, the angles associated to the R, U,L and I codes are respectively equal 
to +1, 2, -1 and 0. Since the sequences of moves considered in this work do 
define U turn, we consider an angle of 2 between two moves as undefined. 

Given a dart d of Gi, let us denote respectively by Fm(d) and Lm(d) the 
moves of the first and last oriented cracks of the segment associated to d. 
If dRFi(d) = d\...d q we have d\ = d and d q = ao(oti(d)) (equation 5) and 
Fm(d) = move(di), Lm(d) = move(d q ). The two darts d\ and d q may thus 
be retrieved in constant time from d. Moreover the moves of d\ and d q are 
retrieved using a correspondence between the oriented cracks and the initial 
darts. This correspondence may be defined using any implicit numbering of 
the initial darts (see e.g. Fig. 3(a)). The values of Fm(d) and Lm(d) may thus 
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be retrieved without additional memory requirement and in constant time 
using an appropriate numbering of the initial darts. 



Given a dart d in Gi, and the sequence of darts d± . . .d q in Go encoding its 
segment, the properties of the segments together with the properties of the 
combinatorial pyramids [10] induce the two following properties: 



where move(dj) denotes the move of the oriented crack associated to dj and 
move{dj)~ 1 is the opposite of the move of dj (e.g. right^ 1 = left). 

Equation 8 states that two successive moves within a segment cannot be op- 
posite. This property is induced by the fact that one segment cannot contain 
twice a same crack with two orientations. Equation 9 states that the first move 
of the Oi successor of a dart d cannot be the opposite of the last move of d. 
Otherwise, the dart d would be an empty self-loop of Gi which is refused by 
hypothesis. 

Given the angle between two successive oriented cracks we define the orienta- 
tion of a dart as the sum of the angles between the oriented cracks along its 
associated segment. Given a dart d in Gi the orientation of d is defined by: 

q-l 

or(d) = 2J (move(dj),move(dj + i))~ (10) 
j'=i 

where d\ . . . d q is the the sequence of initial darts encoding the segment as- 
sociated to d. Note that (move(dj), move(dj+if' cannot be undefined for any 
j G {1, . . . , q — 1} (equation 8). 



Vj G {1, . . . , p — 1} move(dj) ^ move(dj + i) 

Lm(d) ^ Fni^d))- 1 




(9) 
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The orientation of a dart may be computed on demand using equation 10 
or may be attached to each dart and updated during the construction of the 
pyramid. Indeed, let us consider two successive double darts d\ and d 2 at one 
level of the pyramid. If d\ survives at the above level its orientation may be 
updated by [5,7]: 

or(d 1 ) = or(di) + or(d 2 ) + {Lm{d x ) , Fm{d 2 )Y (11) 

Note that this last formula may be extended to the removal of a sequence of 
successive double edges. 

The dart's orientation may thus be computed by fixing the orientation of all 
initial darts to and updating the dart's orientation using equation 11 during 
the removal of each double edge kernel. 

Let us consider a sequence d\...d p in Gi such that dj+i = cn(dj) for all 
j in {1, . . . ,p — 1} and d p ^ «j(<ii). We say that such a sequence defines 
a closed boundary if ai(d p ) and d\ are incident to a same dual vertex e.g. 
d\ G <p*(cxi(dp)). The orientation of such a sequence is defined by: 

p-i 

or(di ...dp) = ^2 (or(dj) + (Lm(^), Fm(d j+1 ))^) + or(d p ) (12) 

The quantity (Lm(d p ) , Fra(4))" has to be added to or(d\ . . .d p ) if the se- 
quence defines a closed boundary. Note that (Lm(dj), Fm(dj + i))^ cannot be 
undefined for any j G {1, . . . ,p — 1} (equation 9). Moreover, one can show 
that if the sequence defines a closed boundary and if Lm(d p ) = Fm{d\)~ 1 , 
then we should have ai(d p ) = d±, which is refused by hypothesis. 

Using the same notations and hypothesis as equation 12, one important result 
shown by Braquelaire and Domenger [5,7] states that the orientation of a 
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sequence d\ . . . ,d p denning a closed boundary is equal to 4 if it is traversed 
clockwise and —4 otherwise. Moreover, this sequence corresponds to: 

• a finite face of Gi and thus a region if its orientation is equal to —4, 

• a set of faces of Gi connected by bridges and contained in one face if the 
orientation is equal to 4. Such a set of faces is called an infinite face [5,7]. 
It encodes a connected component of the partition (Section 1). 

By construction each combinatorial map Gi of a combinatorial pyramid is 
connected and all but one faces of Gi define a finite face. The infinite face of 
a combinatorial map encodes the background of the image (Section 3). 

5 Computing contains/inside relationships 

As demonstrated in Fig. 4, the determination of the contains and inside re- 
lationships requires to determine which vertices are surrounded by a self-loop 
incident to a given vertex. This ambiguity in the location of the self-loop is 
related to the fact that the two darts of a self-loop play a symmetric role 
in the a cycle to which they belong (Section 4.1). The determination of the 
contains and inside relationships requires thus to define a criterion in order 
to differentiate the two darts of a self-loop. This criterion is provided by the 
following proposition (Fig. 5(a)): 

Proposition 1 Consider a combinatorial map Gi defined at level i of a combi- 
natorial pyramid such that Gi does not contain any redundant edge. Let us ad- 
ditionally consider the darts around a vertex o~*(di) = (d±, . . . , dj, . . . , dk, ■ ■ ■ , d p ) 
of Gi and a self-loop ct*(dj) = (dj,dk) such that dart dj is encountered be- 
fore d k when traversing o~*(di) from d 1} e.g. j < k. The two sequences of 
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darts C\ = (dj+i, . . . , dk-\) and C2 = (dfc+i, ■ • • , d p , d\, . . . , dj-i) define closed 
boundaries and have an opposite orientation (or(C\) = —or{C2))- Moreover, 
the two couple of darts (dk-i,dj + i) and (c^+i, dj-i) do not define self-loops 
e.g. d j+1 ^ a>i(d k -i) and dj-i ^ a;(4+i)- 

PROOF. First note that since Gi does not contain empty self-loops both C\ 
and C2 should be non-empty. 

Let us show that C\ defines a closed boundary The definitions of o~*(di) and 
ce*(dj) induce the two following equalities: (pi(oii(dk-i)) = ^{dk-i} = d k and 
<Pi(dk) — o~i(dj) = dj + i. We have thus dj + i = ^{a^dk-x)) which induces 
dj+i G ip*(ai(dk-i))- The same arguments are used to show that C2 defines a 
closed boundary 

Let us now show that dj+i 7^ a^dk-i). Since dj + i = <pf(ai(dk-i)), the relation 
dj + i = ai(dk-i) implies that dj+i = (pf(dj + t). The dart dj + \ would thus be 
incident to a degree two face which is refused by hypothesis since Gi does 
not contain empty double edges. The same argument is used to show that 
dj_i ^ a>i(d k+1 ). 

All the conditions to apply equation 12 are thus satisfied and we derive: 

or(a*(d)) = or{C x ) + or(C 2 ) - 4 (13) 

where or(a*(d)) denotes the orientation of the whole sequence of darts (d±, . . . , d p ). 
Since this sequence defines a counter-clockwise traversal of the face its orien- 
tation is equal to —4 (Section 4.4). We have thus or{C\) = —or{C2)- □ 
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Proposition 1 may be interpreted as follows: The loop a*(dj) corresponds 
to a bridge in Gi the removal of which splits the combinatorial map into 
two connected components. The component encoding the surrounding face is 
traversed counter-clockwise and has thus an orientation of —4. The remaining 
component corresponds to the connected component of inside regions and has 
an opposed orientation of 4 (section 4.4). We say that dj is the starting dart 
of the loop if the sequence of darts encoding the inside connected component 
is enclosed between dj and dk = oti(dj). This property is thus characterized by 
or(Ci) = 4. The dart dj is called the ending dart of the loop otherwise. Note 
that if dj is a starting dart oti(dj) should be an ending dart and conversely. 

The above discussion and Proposition 1 provide thus a criterion which differ- 
entiates the two darts of a loop in order to characterize the inside relationship. 
However, computing the orientation of all sequences of darts between the two 
darts of all self-loops incident to a vertex would require extra calculus. Indeed, 
nested self-loops may induce several traversals of a same sequence of darts. 
The following theorem incrementally computes the orientation of any sequence 
of darts surrounded by the two darts of a loop: 

Proposition 2 Using the same hypothesis and notations as Proposition 1, 
the orientation of the sequence of darts C\ = dj + i . . . .dk-\ between dj and dk 
is defined by : 

or(Ci) = or'(di . . . .dk-i)—or'(dx . . . .dj) — (Lm(dj), Fm(dj + i))>(im(4-i), Fm(dj 

where or'{d\ . . . .dj.-i) and or'(d\ . . . .dj) are the orientations of the sequences 
di . . . .dk-i and di . . . .dj (equation 12) considered as non-closed sequences of 
darts. 
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PROOF. We want to use equation (12) to calculate the orientation of the 
sequences C\ = (dj+i, . . . , dk-i), (di, . . . , dj) and (di, . . . , 4-i)- 

(1) C\. The precondition 4-1 7^ aii(dj + i) is equivalent to dj + i 7^ aj(4-i) 
which is excluded in Proposition 1. Moreover, C\ is a closed sequence. 

(2) (di, . . . , dj): If d\ = cti(dj) then di = d k (since cti(dj) = dk) and thus 
k = 1 and p = k — 1. This last result contradicts our hypothesis j < k. 

(3) (di, . . . , 4-i) : If we assume that 4-i = OLi(d\) and combine it with the 
relationship crj(4-i) = d k = ai(dj) we can express cr*^) as: 



This last equation contradicts the planarity of Gi since the edges a*(d±) 
and 01* (dj) must cross in order to satisfy equation 14 (Fig. 5(b)). 

We now can expand the orientations of the three sequences involved in Proposi- 
tion 2 to show that or (CI)— or'(di, . . . , dk)+or'(di, . . . ,dj) — (Lm(4-i), Fm(dj + i))~— 
(Lm(dj), Fm(dj + i))~. Indeed or(Cl) — or'(d\, . . . , dk) + or'(d\, . . . , dj) may be 
expanded as follows: 



a*(di) = (d u ...,dj,..., d k 




(14) 



(di,... 



. . . , ai(di), ai(dj) ,...,d p ) 



E (or(d r ) + (Lm(d r ) , Fm(d r+1 ))^ 



r=j+l 



+or(d k -i) + (Lm(dk-i),Fm(dj + i)Y 



(or(Cy)) 



- y E (or(d r ) + (Lm(d r ), Fm(d r+1 ))-) + or(4_i)J 
+ ( E or(rf r ) + (Lm(d r ), Fm(d r+1 ))~- (Lm(dj), Fm(d j+ i))~ 




(or'(4, . . • ,4-i)) 



) 



(or'(4, . . . 



(Lm(dk-i),Fm(dj + i)Y- (Lm(dj), Fm(d j+l )Y .□ 
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1 list starting_dart (combijnap Gi, dart d\) { 

2 list L=0 

3 stack P 

4 for each dart dk in cr*(d) = (di, . . . , 

5 if (dfc is a loop) { 

6 if (P is empty or cti(dk) is not on the top of the stack P) 

7 push d^ and or(d\, . . . ,dk) in P 

8 else {// ai(dk) on top of the stack P 

9 let C\ be the sequence of darts between oti(dk) and dk 

10 computes or{C\) using Proposition 2 

11 if (or(Ci) == 4) L = LU{«i(4)} else L = LU{d k } 

12 } 

13 } 

14 return L 

15 } 



Algorithm 1. Determination of the starting darts of the loops 
Propositions 1 and 2 are the basis of the algorithm staring_darts (Algo- 
rithm 1) which traverses the <7j cycle of a given vertex <J*(di) = {d\, . . . , d p ) 
and computes at each step the orientation of the sequence d\ . . . , dk- Using the 
same notations as Proposition 1, let us consider a loop oc*(dj) = (dj,dk) such 
that dj has been previously encountered by the algorithm (j < k). The algo- 
rithm start ing_dart determines the starting dart between dj and dk on lines 
10 and 11 from the orientation of C\ = (dj + i . . . , d^-i) by using Propositions 1 
and 2. This starting dart is added to a list returned by the algorithm. 

Since the loops are nested dj and or'(d\ dj) should be on the top of stack 

P used by the algorithm. The darts dj,dj + i and dk-i are retrieved from the 



26 



current dart dk by: dj = ai(dk) ; dj + i = o~i(dj) and dk-i = a^ x (dk). More- 
over, the orientation of C\ (Proposition 2) is evaluated in constant time since 
or'(di, . . . , dk-i) is the last orientation and or'(d\ . . . ,dj) is retrieved from the 
stack. 

Given the list of starting darts determined by the algorithm stating_darts, 
the set of vertices contained in c*(d\) is retrieved by traversing, the sequence 
(7*{di) from each starting dart to the corresponding end. By construction all 
darts encountered between the starting and ending darts of the loop encode 
adjacency relationships to contained vertices. Note that in case of nested loops 
some loops may be traversed several times. Given a starting dart d, this last 
drawback may be avoided by replacing any encountered starting dart by its 
a.i successor during the traversal from d to oti{d). 

Our algorithm, is thus local to each vertex and the method may be applied 
in parallel to all the vertices of the combinatorial map Gj. Given a vertex 
a*(di), the determination of its starting darts (algorithm starting_darts) 
requires to traverse once a*(d\). Moreover, the determination of the inside 
relationships from the list of starting darts requires to traverse each dart of 
<r*(di) at most once. The worse complexity of our algorithm is thus bounded 
by the maximum degree of a vertex, e.g. 0(2\a*(di)\). 

5.1 Application to road sign's recognition 

Fig. 6 illustrates one application of the contains/inside information to image 
analysis. The road sign shown in Fig. 6(a) is composed of only two colors 
with one symbol inside a uniform background, the background itself being 
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surrounded by one border with a same color as the symbol. In our example, 
the two roadsigns have a uniform background which includes one symbol rep- 
resenting a white arrow. The background is surrounded by a white border. In 
this application we wish to extract the sign using only topological and color 
information (and thus independently of the shapes of the symbol and the road 
sign). Using only adjacency and color information, the symbol cannot be dis- 
tinguished from the border of the road sign since the border and the symbol 
have a same color and are both adjacent to the background of the road sign 
(Fig. 6(d)). However, using contains/inside information, the symbol and the 
border may be distinguished since the border contains the background of the 
road sign which contains the symbol. Our algorithm first builds a combina- 
torial pyramid using a hierarchical watershed algorithm [15]. Fig. 6(b) shows 
the top level of the hierarchies obtained from the two roadsigns. Using the top 
level combinatorial map of each pyramid our algorithm selects the k regions 
of the partition whose color is closest from the background's color (k is fixed 
to five in our experiment). This last step defines a set of candidate regions for 
the background of the road sign. This background is then determined as the 
region whose contained regions have the closest mean color from the color's 
symbol (equal to white in this experiment). Note that this step removes from 
the k selected candidates any regions which do not contain another region. 
We thus make explicit the a priori knowledge that the background of the road 
sign should contain at least one region. The symbol is then determined as 
the set of regions inside the selected region (Fig. 6(c)). Finally, let us note 
that the contains information needs to be computed only on the k selected 
candidates for the road sign's background. Within this experiment a global 
algorithm computing the contains information for all vertices would require 
useless calculus. 
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6 Conclusion 



We have introduced in this paper 5 relationships between regions (Section 1.1). 
These relationships are devoted to the graph based segmentation framework 
and encode either rough or fine relationships between the regions of a parti- 
tion: The meets_exists relationship corresponds to the ability of a model to 
encode the existence of at least one common boundary between two regions. 
The meets-each relationship corresponds to an encoding of each connected 
boundary between two adjacent regions. The inside and contains relation- 
ships are asymmetric and encode the fact that one region contains the other. 
Finally, the composed of relationships is only provided by hierarchical data 
structures and encodes the fact that one region is composed of several regions 
defined at levels below. 

Table ?? shows the ability of the Region Adjacency Graph, the combinatorial 
map, the simple graph pyramid, the dual graph pyramid and the combinatorial 
pyramid to encode the meets -exist, meets-each, contains/inside and composed 
of relationships. The Region Adjacency Graph (Section 1.2) only encodes the 
meets-each relationship. The combinatorial map model (Section 1.3) encodes 
all but the composed of relationships. The simple graph pyramids (Section 2) 
encodes the meets-exist and the composed of relationships. This last rela- 
tionship is also encoded by the two other irregular pyramid models described 
in this paper (section 1.4). The dual graph pyramids (Section 2.2) encodes 
the meets-exists, meets-each (Section 2.3) and composed of relationships. The 
inside/ contains relationships can not be deduced from the model using local 
calculus (Section 2.4). However, the authors conjecture that these relationships 
may be computed using the fact that the vertex encoding the background of 
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the image is not surrounded by any self-loop. Such an algorithm would require 
a propagation step from the background vertex and would thus require global 
calculus. This property is indicated by an interrogation mark in Table ??. 
The combinatorial map pyramid model (Section 4) encodes the meets -exists, 
meets-each (Section 4.1) and composed of relationships. 

The main contribution of this paper consists in the design of the algorithm 
starting_dart (Section 5) which uses the orientation explicitly encoded by 
combinatorial maps to differentiate the two darts of a self-loop. Given a vertex 
incident to a self-loop, this last characterization allows to determine the regions 
inside the region encoded by this vertex in a time proportional to twice its 
number of incident edges. This method implies only local calculus and its 
parallel complexity is bounded by twice the maximal degree of the vertices of 
the graph. 

The efficient computation of those relations relating regions of a segmentation 
is a prerequisite to the description and the recognition of relevant groupings: 
an important step on the way to more generic recognition, categorization and 
higher visual abstraction within the homogeneous framework of combinatorial 
pyramids. 
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(a) (b) (c) RAG(a)=RAG(b) 

Fig. 1. The ideal segmentation of the two roadsigns (a) and (b) are encoded by the 
same RAG (c). 
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Fig. 2. The Graph (b) defines the top of a dual graph pyramid encoding an ideal 
segmentation of (a). The self loop incident to vertex A may surround either vertex 
B or C without changing the incidence relations between vertices and faces. The 
dual vertices associated to faces are represented by filled boxes (■). Dual edges are 
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Fig. 3. A 3 x 3 grid encoded by a combinatorial map 
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(a) (b) 



Fig. 4. The encoding of an ideal segmentation of a road sign (a) by the top level 
combinatorial map of a pyramid may be drawn using either (b) or (c). 
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Fig. 5. The local configuration in Gi (a) and Gj(b) of the darts used by Propositions 1 
and 2. Note that we implicitly suppose here that dj is the starting dart since C2 
surrounds C\. (c) A contradiction obtained in the proof of Proposition 2. 




